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Is There Something Else Wrong with the FCAT? 


A Closer Look at Reading Gains in Middle School 


The Precedent for Concern 

Because 3rd Grade FCAT scores have a direct impact on promotion, the results for that grade level are 
released early by the State. When the FCAT results for 3rd Grade were released in May 2007, many people 
were troubled. Over 80% of the elementary schools in the Miami-Dade School District showed a decrease in 
the percentage of students scoring in Levels 3 and above in Reading. Similar trends produced an outcry 
across the state. After an 8 percentage point jump in the previous year, there was a statewide drop of 6 
percentage points in 2007. These results were more than surprising and cast doubt on the soundness of 
the scoring process. After a careful review, the Education Commissioner announced that there did appear 
to be a problem in the scoring of the test and a rescaling of the 3rd Grade FCAT reading exam would be 
forthcoming. 

This admission by the State, while honorable and praiseworthy, opened the door for many to question other 
aspects of the FCAT test, sometimes without just cause. It is apparent that, despite every effort to ensure 
the integrity and accuracy of the test, scores can be invalid and misleading. Even so, we should not let this 
disclosure by the State lead us to abuse our right of inquiry, blaming any unflattering test result on scoring 
error. 

While it is clearly unfair to discount the overall precision and usefulness of the FCAT, some form of careful 
review of scores is nonetheless proper. The search for the same kind of irregular trends in the data that 
brought this type of problem to the surface is always warranted and should be welcomed. Any abnormalities 
revealed in the examination of the scores need to be made public so that the need for in-depth analysis can 
be considered. It is in that spirit that this paper presents some unusual patterns in the data for FCAT 
Reading at the middle school level. 

Beyond the Changes in School Grades 

In 2007, the State changed the way it determined the grades for schools since, for the first time, two new 
components were considered. These new components included the percent of students scoring in levels 3 
and above in science and the percent of students in the lowest 25% making gains in math. These additions 
meant a rescaling of the points needed for schools to achieve the various grade designations. Largely as a 
result of these and other grading scheme changes, the grades for schools this year showed a marked 
decline. For example, the number of ‘F’ schools increased from 21 last year to 82 this year across the state. 
While this type of decline in school grades may be typical when new factors are incorporated, it makes the 
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grade distributions between the years no longer directly comparable. If we intend to investigate unusual 
data patterns, we must be sure that our observations are not confused with changes in the grading scheme. 

For this reason, this study was confined to the grade components common across all of the last three years. 
Specifically, only those aspects of the grading scheme were considered in which schools were awarded one 
point for every percent of students (1) scoring in achievement levels 3-5 in Reading, (2) demonstrating 
learning gains in Reading, (3) in the lowest 25% of Reading making gains, (4) scoring in achievement levels 
3-5 in Mathematics, (5) demonstrating learning gains in Mathematics, and (6) scoring 3.5 or above in Writing. 

Average Changes in Grade Components 

As a first step in monitoring the test performance in the district, it was reasonable to look at changes between 
last year and this year in the points awarded for each of the six consistently-used components of the school 
grading scheme. 
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From this simple look at the data, two cells stand out. Compared to the other components and other school 
levels, Reading Gains and Lowest 25% Reading showed dramatic losses at the Middle school level. Note 
that both of these components are derived from the Reading test and both are gains-between-years compared 
to last year’s gains-between-years. These surprising findings naturally prompt the next question: Are these 
differences found at extreme levels in only a few middle schools or are they consistent throughout the 
district? 

Changes in Reading Gains by Schooi 

The graph below presents the difference in Reading Gains for each school in the district. The Elementary 
schools show mixed results, with most increasing their percent of students showing Reading Gains and 
some decreasing. Senior High schools are less positive. But Middle schools consistently show decline with 
only two schools showing increases (and one of those, the highest, is explainable by changes in the grade 
structure). 
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It’s hard to believe that almost all of our Middle schools suddenly did worse this year in teaching reading 
skills. This graph represents changes in Reading Gains percentages; the graph for differences in the Lowest 
25% of the Students making gains in Reading would look so similar that it is not necessary to repeat the 
picture here. These results lead to the next question: Is this a phenomenon only in Miami-Dade schools? 

Changes Across the State 

In the following graph all of the schools in the state are depicted. It can be seen here that the same pattern 
is observed throughout the state. Less than 8% of the Middle schools in the entire state show increases in 
Reading Gains (and the great majority of those gained only one or two points). Once again, the picture for 
Gains in the Lowest 25% is strikingly similar. 
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Where to Go From Here 

On this scale it now defies credulity that virtually all of the Middle schools across the state have somehow 
dropped the ball when it comes to teaching Reading. The logical conclusion is that something strange has 
happened in the tests. 

The test for each grade level in Middle school should be examined. Because these gain measures involve 
changes between years, it is difficult to pinpoint any particular faulty test. It may be that the apparent gains 
across the previous two years were “inflated” in a manner similar to the problem involving 3rd grade. For 
instance, the change in the percent of students scoring at levels 3 and above jumped 8 percent for grade 
3 (from 67% to 75%) between 2005 and 2006. This kind of performance disparity alerted the state to 
possible problems with equating test difficulty. It is worth noting that the change in percent of students 
scoring at level 3 and above between the same years was also 8 percent for grade 6 (from 56% to 64%) 
and grade 7 (from 53% to 64%). Dramatic inflated gains in 2006 might explain the almost total absence of 
gains observed in 2007. 

Without access to item-level data on the FCAT, districts can only speculate about potential scoring problems 
and their effects on setting cutscores and grade designations. Clearly no test is perfect and, no matter how 
carefully constructed, standardized tests can not exactly reflect academic progress. Suspending judgement 
about any wrongdoing, it is reasonable to call upon the State to explain these anomalies at the Middle 
school level in Reading. 
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